Status of the health information system in Ireland and its fitness to support health system performance assessment: a multimethod assessment based on stakeholder involvement.

BACKGROUND: Between 2019 and 2021, the first Irish health system performance assessment (HSPA) framework was developed. As routinely collected health data are necessary to continuously populate indicators of an HSPA framework, a purpose-driven assessment of the health information system (HIS) in Ireland and its fitness to support the implementation of an HSPA framework was conducted. This study reports on the status of the Irish HIS through a multimethod assessment based on continuous broad stakeholder involvement. METHODS: Between May and November 2020, over 50 informants were engaged in individual and group interviews and stakeholder consultation workshops as part of the HIS assessment process. Descriptive themes and high-level data availability heatmaps were derived from interview and workshop data using thematic analysis. Indicator "passports" for the HSPA framework were populated during stakeholder consultation workshops and analysed using univariate descriptive statistics. RESULTS: The HIS in Ireland was able to provide administrative, survey and registry-based data for public sector acute care services, focusing on structure, process and output metrics. Significant data availability gaps, most notably from primary care, private hospitals and community care, were reported, with little availability of electronic health record and people-reported data. Data on outcome metrics were mostly missing, as were linkage possibilities across datasets for care pathway monitoring. The COVID-19 pandemic highlighted the national HIS's shortcomings but also the capacity for rapid development and improvement. CONCLUSIONS: A tailor-made assessment of the HIS in Ireland, involving a broad set of relevant stakeholders, revealed strengths, weaknesses and areas for improvement in the Irish health data landscape. It also contributed to the development of a national HSPA framework and momentum to further strengthen data infrastructure and governance, while working towards a more data-driven and person-centred healthcare system. This work demonstrates the utility of an inclusive HIS assessment process and is applicable beyond Ireland, where this case study was conducted.

The experiences of 33 national COVID-19 dashboard teams during the first year of the pandemic in the World Health Organization European Region: A qualitative study.

Background: Governments across the World Health Organization (WHO) European Region have prioritised dashboards for reporting COVID-19 data. The ubiquitous use of dashboards for public reporting is a novel phenomenon. Objective: This study explores the development of COVID-19 dashboards during the first year of the pandemic and identifies common barriers, enablers and lessons from the experiences of teams responsible for their development. Methods: We applied multiple methods to identify and recruit COVID-19 dashboard teams, using a purposive, quota sampling approach. Semi-structured group interviews were conducted from April to June 2021. Using elaborative coding and thematic analysis, we derived descriptive and explanatory themes from the interview data. A validation workshop was held with study participants in June 2021. Results: Eighty informants participated, representing 33 national COVID-19 dashboard teams across the WHO European Region. Most dashboards were launched swiftly during the first months of the pandemic, February to May 2020. The urgency, intense workload, limited human resources, data and privacy constraints and public scrutiny were common challenges in the initial development stage. Themes related to barriers or enablers were identified, pertaining to the pre-pandemic context, pandemic itself, people and processes and software, data and users. Lessons emerged around the themes of simplicity, trust, partnership, software and data and change. Conclusions: COVID-19 dashboards were developed in a learning-by-doing approach. The experiences of teams reveal that initial underpreparedness was offset by high-level political endorsement, the professionalism of teams, accelerated data improvements and immediate support with commercial software solutions. To leverage the full potential of dashboards for health data reporting, investments are needed at the team, national and pan-European levels.

Changes in the quality of cancer care as assessed through performance indicators during the first wave of the COVID-19 pandemic in 2020: a scoping review.

BACKGROUND: Cancer comprises a high burden on health systems. Performance indicators monitoring cancer outcomes are routinely used in OECD countries. However, the development of process and cancer-pathway based information is essential to guide health care delivery, allowing for better monitoring of changes in the quality of care provided. Assessing the changes in the quality of cancer care during the COVID-19 pandemic requires a structured approach considering the high volume of publications. This study aims to summarize performance indicators used in the literature to evaluate the impact of the COVID-19 pandemic on cancer care (January-June 2020) in OECD countries and to assess changes in the quality of care as reported via selected indicators. METHODS: Search conducted in MEDLINE and Embase databases. Performance indicators and their trends were collated according to the cancer care pathway. RESULTS: This study included 135 articles, from which 1013 indicators were retrieved. Indicators assessing the diagnostic process showed a decreasing trend: from 33 indicators reporting on screening, 30 (91%) signalled a decrease during the pandemic (n = 30 indicators, 91%). A reduction was also observed in the number of diagnostic procedures (n = 64, 58%) and diagnoses (n = 130, 89%). The proportion of diagnoses in the emergency setting and waiting times showed increasing trends (n = 8, 89% and n = 14, 56%, respectively). A decreasing trend in the proportion of earliest stage cancers was reported by 63% of indicators (n = 9), and 70% (n = 43) of indicators showed an increasing trend in the proportion of advanced-stage cancers. Indicators reflecting the treatment process signalled a reduction in the number of procedures: 79%(n = 82) of indicators concerning surgeries, 72%(n = 41) of indicators assessing radiotherapy, and 93%(n = 40) of indicators related to systemic therapies. Modifications in cancer treatment were frequently reported: 64%(n = 195) of indicators revealed changes in treatment. CONCLUSIONS: This study provides a summary of performance indicators used in the literature to assess the cancer care pathway from January 2020 to June 2020 in OECD countries, and the changes in the quality of care signalled by these indicators. The trends reported inform on potential bottlenecks of the cancer care pathway. Monitoring this information closely could contribute to identifying moments for intervention during crises.

Exploring changes to the actionability of COVID-19 dashboards over the course of 2020 in the Canadian context: Descriptive assessment and expert appraisal study

Background: Public web-based COVID-19 dashboards are in use worldwide to communicate pandemic-related information. Actionability of dashboards, as a predictor of their potential use for data-driven decision-making, was assessed in a global study during the early stages of the pandemic. It revealed a widespread lack of features needed to support actionability. In view of the inherently dynamic nature of dashboards and their unprecedented speed of creation, the evolution of dashboards and changes to their actionability merit exploration. Objective: We aimed to explore how COVID-19 dashboards evolved in the Canadian context during 2020 and whether the presence of actionability features changed over time. Methods: We conducted a descriptive assessment of a pan-Canadian sample of COVID-19 dashboards (N = 26), followed by an appraisal of changes to their actionability by a panel of expert scorers (N = 8). Scorers assessed the dashboards at two points in time, July and November 2020, using an assessment tool informed by communication theory and health care performance intelligence. Applying the nominal group technique, scorers were grouped in panels of three, and evaluated the presence of the seven defined features of highly actionable dashboards at each time point. Results: Improvements had been made to the dashboards over time. These predominantly involved data provision (specificity of geographic breakdowns, range of indicators reported, and explanations of data sources or calculations) and advancements enabled by the technologies employed (customization of time trends and interactive or visual chart elements). Further improvements in actionability were noted especially in features involving local-level data provision, time-trend reporting, and indicator management. No improvements were found in communicative elements (clarity of purpose and audience), while the use of storytelling techniques to narrate trends remained largely absent from the dashboards. Conclusions: Improvements to COVID-19 dashboards in the Canadian context during 2020 were seen mostly in data availability and dashboard technology. Further improving the actionability of dashboards for public reporting will require attention to both technical and organizational aspects of dashboard development. Such efforts would include better skill-mixing across disciplines, continued investment in data standards, and clearer mandates for their developers to ensure accountability and the development of purpose-driven dashboards. (PsycInfo Database Record (c) 2022 APA, all rights reserved)

The Impact of the COVID-19 Pandemic on Hospital Services for Patients with Cardiac Diseases: A Scoping Review.

This study aims to assess the impact of the COVID-19 pandemic on hospital cardiac care, as assessed by performance indicators. Scoping review methodology: performance indicators were extracted to inform on changes in care during January-June 2020. Database searches yielded 6277 articles, of which 838 met the inclusion criteria. After full-text screening, 94 articles were included and 1637 indicators were retrieved. Most of the indicators that provided information on changes in the number of admissions (n = 118, 88%) signaled a decrease in admissions; 88% (n = 15) of the indicators showed patients' delayed presentation and 40% (n = 54) showed patients in a worse clinical condition. A reduction in diagnostic and treatment procedures was signaled by 95% (n = 18) and 81% (n = 64) of the indicators, respectively. Length of stay decreased in 58% (n = 21) of the indicators, acute coronary syndromes treatment times increased in 61% (n = 65) of the indicators, and outpatient activity decreased in 94% (n = 17) of the indicators related to outpatient care. Telehealth utilization increased in 100% (n = 6). Outcomes worsened in 40% (n = 35) of the indicators, and mortality rates increased in 52% (n = 31). All phases of the pathway were affected. This information could support the planning of care during the ongoing pandemic and in future events.

Engaging citizens in the development of a health system performance assessment framework: a case study in Ireland.

BACKGROUND: The launch in 2017 of the Irish 10-year reform programme Sláintecare represents a key commitment in the future of the health system. An important component of the programme was the development of a health system performance assessment (HSPA) framework. In 2019, the Department of Health of Ireland (DoH) and Health Service Executive (HSE) commissioned the technical support of researchers to develop an outcome-oriented HSPA framework which should reflect the shared priorities of multiple stakeholders, including citizens. This study describes the method applied in the Irish context and reflects on the added value of using a citizen panel in the development of an HSPA framework. METHODS: A panel of 15 citizens was convened, recruited by a third-party company using a sampling strategy to achieve a balanced mix representing the Irish society. Panellists received lay-language preparatory materials before the meeting. Panellists used a three-colour scheme to signal the importance of performance measures. An exit questionnaire was administered to understand how participants experienced being part of the panel. The citizen panel was the first in a series of three panels towards the development of the HSPA framework, followed by panels including representatives of the DoH and HSE, and representatives from professional associations and special interest groups. RESULTS: The citizen panel generated 249 health performance measures ranging across 13 domains. Top-ranking domains to the citizen panel (people-centredness, coordination of care, and coverage) were less prioritized by the other panels; domains less prioritized by the citizen panel, such as accessibility, responsiveness, efficiency, and effectiveness, were of higher priority in the other panels. Citizen panellists shared a similar understanding of what a citizen panel involves and described their experience at the panel as enjoyable, interesting, and informative. CONCLUSIONS: The priorities of the citizen panel were accounted for during all phases of developing the HSPA framework. This was possible by adopting an inclusive development process and by engaging citizens early on. Citizen engagement in HSPA development is essential for realizing citizen-driven healthcare system performance and generating trust and ownership in performance intelligence. Future research could expand the use of citizen panels to assess, monitor, and report on the performance of healthcare systems.

Features Constituting Actionable COVID-19 Dashboards: Descriptive Assessment and Expert Appraisal of 158 Public Web-Based COVID-19 Dashboards.

BACKGROUND: Since the outbreak of COVID-19, the development of dashboards as dynamic, visual tools for communicating COVID-19 data has surged worldwide. Dashboards can inform decision-making and support behavior change. To do so, they must be actionable. The features that constitute an actionable dashboard in the context of the COVID-19 pandemic have not been rigorously assessed. OBJECTIVE: The aim of this study is to explore the characteristics of public web-based COVID-19 dashboards by assessing their purpose and users ("why"), content and data ("what"), and analyses and displays ("how" they communicate COVID-19 data), and ultimately to appraise the common features of highly actionable dashboards. METHODS: We conducted a descriptive assessment and scoring using nominal group technique with an international panel of experts (n=17) on a global sample of COVID-19 dashboards in July 2020. The sequence of steps included multimethod sampling of dashboards; development and piloting of an assessment tool; data extraction and an initial round of actionability scoring; a workshop based on a preliminary analysis of the results; and reconsideration of actionability scores followed by joint determination of common features of highly actionable dashboards. We used descriptive statistics and thematic analysis to explore the findings by research question. RESULTS: A total of 158 dashboards from 53 countries were assessed. Dashboards were predominately developed by government authorities (100/158, 63.0%) and were national (93/158, 58.9%) in scope. We found that only 20 of the 158 dashboards (12.7%) stated both their primary purpose and intended audience. Nearly all dashboards reported epidemiological indicators (155/158, 98.1%), followed by health system management indicators (85/158, 53.8%), whereas indicators on social and economic impact and behavioral insights were the least reported (7/158, 4.4% and 2/158, 1.3%, respectively). Approximately a quarter of the dashboards (39/158, 24.7%) did not report their data sources. The dashboards predominately reported time trends and disaggregated data by two geographic levels and by age and sex. The dashboards used an average of 2.2 types of displays (SD 0.86); these were mostly graphs and maps, followed by tables. To support data interpretation, color-coding was common (93/158, 89.4%), although only one-fifth of the dashboards (31/158, 19.6%) included text explaining the quality and meaning of the data. In total, 20/158 dashboards (12.7%) were appraised as highly actionable, and seven common features were identified between them. Actionable COVID-19 dashboards (1) know their audience and information needs; (2) manage the type, volume, and flow of displayed information; (3) report data sources and methods clearly; (4) link time trends to policy decisions; (5) provide data that are "close to home"; (6) break down the population into relevant subgroups; and (7) use storytelling and visual cues. CONCLUSIONS: COVID-19 dashboards are diverse in the why, what, and how by which they communicate insights on the pandemic and support data-driven decision-making. To leverage their full potential, dashboard developers should consider adopting the seven actionability features identified.

Development and Actionability of the Dutch COVID-19 Dashboard: Descriptive Assessment and Expert Appraisal Study.

BACKGROUND: Web-based public reporting by means of dashboards has become an essential tool for governments worldwide to monitor COVID-19 information and communicate it to the public. The actionability of such dashboards is determined by their fitness for purpose-meeting a specific information need-and fitness for use-placing the right information into the right hands at the right time and in a manner that can be understood. OBJECTIVE: The aim of this study was to identify specific areas where the actionability of the Dutch government's COVID-19 dashboard could be improved, with the ultimate goal of enhancing public understanding of the pandemic. METHODS: The study was conducted from February 2020 to April 2021. A mixed methods approach was carried out, using (1) a descriptive checklist over time to monitor changes made to the dashboard, (2) an actionability scoring of the dashboard to pinpoint areas for improvement, and (3) a reflection meeting with the dashboard development team to contextualize findings and discuss areas for improvement. RESULTS: The dashboard predominantly showed epidemiological information on COVID-19. It had been developed and adapted by adding more in-depth indicators, more geographic disaggregation options, and new indicator themes. It also changed in target audience from policy makers to the general public; thus, a homepage was added with the most important information, using news-like items to explain the provided indicators and conducting research to enhance public understanding of the dashboard. However, disaggregation options such as sex, socioeconomic status, and ethnicity and indicators on dual-track health system management and social and economic impact that have proven to give important insights in other countries are missing from the Dutch COVID-19 dashboard, limiting its actionability. CONCLUSIONS: The Dutch COVID-19 dashboard developed over time its fitness for purpose and use in terms of providing epidemiological information to the general public as a target audience. However, to strengthen the Dutch health system's ability to cope with upcoming phases of the COVID-19 pandemic or future public health emergencies, we advise (1) establishing timely indicators relating to health system capacity, (2) including relevant data disaggregation options (eg, sex, socioeconomic status), and (3) enabling interoperability between social, health, and economic data sources.

COVID-19 Preparedness and Perceived Safety in Nursing Homes in Southern Portugal: A Cross-Sectional Survey-Based Study in the Initial Phases of the Pandemic.

(1) Background: Nursing homes' preparedness in managing a public health emergency has been poor, with effects on safety culture. The objective of this study was to assess nursing homes' COVID-19 preparedness in southern Portugal, including staff's work experiences during the pandemic. (2) Methods: We used a COVID-19 preparedness checklist to be completed by management teams, followed by follow-up calls to nursing homes. Thereafter, a survey of staff was applied. Data analysis included descriptive statistics, exploratory factor analysis, and thematic analysis of open-end questions. (3) Results: In total, 71% (138/195) of eligible nursing homes returned the preparedness checklist. We conducted 83 follow-up calls and received 720 replies to the staff survey. On average, 25% of nursing homes did not have an adequate decision-making structure to respond to the pandemic. Outbreak capacity and training were areas for improvement among nursing homes' contingency plans. We identified teamwork as an area of strength for safety culture, whereas compliance with procedures and nonpunitive response to mistakes need improvement. (4) Conclusions: To strengthen how nursing homes cope with upcoming phases of the COVID-19 pandemic or future public health emergencies, nursing homes' preparedness and safety culture should be fostered and closely monitored.

Exploring Changes to the Actionability of COVID-19 Dashboards Over the Course of 2020 in the Canadian Context: Descriptive Assessment and Expert Appraisal Study.

BACKGROUND: Public web-based COVID-19 dashboards are in use worldwide to communicate pandemic-related information. Actionability of dashboards, as a predictor of their potential use for data-driven decision-making, was assessed in a global study during the early stages of the pandemic. It revealed a widespread lack of features needed to support actionability. In view of the inherently dynamic nature of dashboards and their unprecedented speed of creation, the evolution of dashboards and changes to their actionability merit exploration. OBJECTIVE: We aimed to explore how COVID-19 dashboards evolved in the Canadian context during 2020 and whether the presence of actionability features changed over time. METHODS: We conducted a descriptive assessment of a pan-Canadian sample of COVID-19 dashboards (N=26), followed by an appraisal of changes to their actionability by a panel of expert scorers (N=8). Scorers assessed the dashboards at two points in time, July and November 2020, using an assessment tool informed by communication theory and health care performance intelligence. Applying the nominal group technique, scorers were grouped in panels of three, and evaluated the presence of the seven defined features of highly actionable dashboards at each time point. RESULTS: Improvements had been made to the dashboards over time. These predominantly involved data provision (specificity of geographic breakdowns, range of indicators reported, and explanations of data sources or calculations) and advancements enabled by the technologies employed (customization of time trends and interactive or visual chart elements). Further improvements in actionability were noted especially in features involving local-level data provision, time-trend reporting, and indicator management. No improvements were found in communicative elements (clarity of purpose and audience), while the use of storytelling techniques to narrate trends remained largely absent from the dashboards. CONCLUSIONS: Improvements to COVID-19 dashboards in the Canadian context during 2020 were seen mostly in data availability and dashboard technology. Further improving the actionability of dashboards for public reporting will require attention to both technical and organizational aspects of dashboard development. Such efforts would include better skill-mixing across disciplines, continued investment in data standards, and clearer mandates for their developers to ensure accountability and the development of purpose-driven dashboards.